Information aggregation

Information aggregation is the process by which knowledge gathered in separate places becomes useful to a group as a whole. The book’s central distinction is that aggregation is not the same as centralization: a system can preserve local expertise and independent judgment while creating ways to combine them. Without such connections, decentralization leaves valuable knowledge stranded; with them, a group can make a more informed judgment than any one participant could make alone.

Local knowledge and the aggregation problem

Decentralization allows people close to a problem to use tacit knowledge: insight rooted in a particular job, place, or experience that is difficult to summarize and transmit. It also promotes specialization and independent judgment, increasing the amount and variety of information available to a group. But this strength creates a corresponding weakness. Information discovered in one part of a system may never reach the others, so the system’s total knowledge does not become shared understanding.

The task is therefore to preserve the specificity of local knowledge while making it collectively useful. The book compares this challenge across markets, corporations, and intelligence agencies. The goal is not to force every local observation into a single authority’s hands, but to build mechanisms through which dispersed knowledge and judgments can be brought together without abolishing the autonomy that generated them.

Intelligence without a central bottleneck

The failure of the United States intelligence community to anticipate major terrorist attacks from 1993 through 2001 illustrates the difference between decentralization and aggregation. Congressional criticism focused on poor information sharing: agencies held localized snapshots rather than producing a coherent picture of threats. Senator Richard Shelby argued that the Federal Bureau of Investigation’s decentralized structure had fragmented information into largely independent fiefdoms, and called for an agency above the disputatious bureaucracies to take charge.

The book accepts the need to share information but rejects the assumption that a small central group must interpret it all. Different agencies’ specializations can reveal different aspects of a problem; information one set of analysts regards as noise may offer another set a critical clue. What was missing was a way to aggregate not only data but also the judgments of the National Security Agency, the Central Intelligence Agency, and the Federal Bureau of Investigation. Centralizing authority might improve coordination while sacrificing diversity, independence, and local knowledge. The proposed alternative is aggregation: connect the agencies’ information and judgments while allowing them to retain their distinct roles. Even a shared database or unified watch list could have helped; two years after September 11, the government still lacked a watch list drawing on data from the whole intelligence community.

Distributed collaboration in science

The World Health Organization’s SARS investigation shows how aggregation can work without a commanding center. In March 2003, the WHO brought eleven laboratories from countries including China, the United States, France, Germany, and Japan into a collaborative research effort. The laboratories held daily teleconferences, posted images and analyses online, and exchanged virus samples so they could check and build on one another’s work. They pursued overlapping lines of inquiry, quickly ruling out possible causes and testing promising ones.

On March 21, researchers at Hong Kong University and the Centers for Disease Control independently isolated candidate viruses. Laboratories then detected a coronavirus in samples from SARS patients, sequenced it, and tested its effects. By April 16, the network had concluded that the coronavirus caused SARS. The discovery cannot be assigned to one person: Cynthia Goldsmith first spotted the coronavirus, but the international group’s subsequent work established that it caused the disease. The WHO convened the collaboration, yet did not dictate each laboratory’s research or allocate its tasks. Its members shared relevant findings and spoke each morning, then worked out the division of labor themselves. The result was collective discovery in weeks rather than the months or years a laboratory might have needed alone.

This case reflects a broader scientific practice: researchers publish results and make data available so others can scrutinize, challenge, and extend them. Failed hypotheses can contribute by narrowing the field of inquiry, and each scientist relies on knowledge produced by others. Recognition, too, is collective: findings become part of scientific knowledge as other researchers assess and accept them. The coronavirus caused SARS before the WHO announcement, but its status as an established scientific finding depended on the community’s scrutiny and provisional consensus. Most scientists cannot personally replicate every result; scientific work therefore relies on trust as well as reproducibility, alongside the possibility of later replication and the exposure of fraud.

Balancing autonomy with shared judgment

The book treats information aggregation as a problem of organizational design, not simply information storage. A firm or agency may collect many local observations yet remain unintelligent if it has no process for combining them. Conversely, a hierarchy that channels every decision upward can suppress the knowledge it is meant to gather. At a General Motors plant in Van Nuys, a worker had spent two years welding a bracket without knowing that it held the car’s sunshade, an example of how information can fail to travel even within a single workplace. In another company, a new product idea had to traverse 223 possible bureaucratic paths. Such barriers make it difficult for workers’ knowledge to reach decision-makers and for executives to know whether their picture of the organization matches reality.

The practical implication is to make local decisions locally when possible, while creating channels that carry useful knowledge across the organization. People closest to a problem may be better positioned than managers to devise a workable solution, and giving employees authority over their working conditions can improve engagement and performance. The aim is neither uncoordinated fragmentation nor command from the top, but an arrangement in which specialization and local responsibility coexist with effective communication and collective judgment.

Markets and the limits of centralization

The book’s account of Friedrich Hayek’s local-knowledge argument clarifies both the value and the limits of decentralized systems. Hayek stressed that markets coordinate knowledge that no central planner can readily condense. The author accepts the force of that argument but contends that skepticism about centralized authority can lead to underestimating the benefits of aggregation. A pari-mutuel betting market, for example, has a central authority taking the other side of bets and producing shared odds; despite this centralized feature, it can offer an eerily accurate picture of the future.

Information aggregation, then, does not require that all knowledge be converted into one person’s judgment. It requires mechanisms that let dispersed contributions inform a common result while preserving the independence and local understanding on which those contributions depend. The intelligence community’s fragmented files, the laboratories’ shared investigation, and the obstacles inside hierarchical companies illustrate the same underlying tension: information can be abundant and still fail to become collective intelligence unless the system allows it to travel and combines what its participants know.